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1.0  EXECUTIVE  SUMMARY 


1.1  BACKGROUND 

Munitions  and  explosives  of  concern  (MEC)  contamination  is  a  high  priority  problem  for  the 
Department  of  Defense  (DoD).  Recent  DoD  estimates  of  MEC  contamination  across 
approximately  1,400  DoD  sites  indicate  that  10  million  acres  are  suspected  of  containing  MEC. 
Because  many  sites  are  large  in  size  (greater  than  10,000  acres),  the  investigation  and 
remediation  of  these  sites  could  cost  billions  of  dollars.  However,  on  many  of  these  sites  only  a 
small  percentage  of  the  site  may  in  fact  contain  MEC  contamination.  Therefore,  determining 
applicable  technologies  to  define  the  contaminated  areas  requiring  further  investigation  and 
munitions  response  actions  could  provide  significant  cost  savings.  Therefore,  the  Defense 
Science  Board  (DSB)  has  recommended  further  investigation  and  use  of  wide  area  assessment 
(WAA)  technologies  to  address  the  potential  these  technologies  offer  in  terms  of  determining  the 
actual  extent  of  MEC  contamination  on  DoD  sites  (DSB,  2003). 

In  response  to  the  DSB  Task  Eorce  report  and  recent  congressional  interest,  the  Environmental 
Security  Technology  Certification  Program  (ESTCP)  designed  a  WAA  pilot  program  that 
consists  of  demonstrations  at  multiple  sites  to  validate  the  application  of  a  number  of  recently 
developed  and  validated  technologies  as  a  comprehensive  approach  to  WAA.  These 
demonstrations  of  WAA  technologies  include  deployment  of  high  airborne  sensors,  helicopter- 
home  magnetometry  arrays  and  ground  surveys. 

This  report  describes  the  cost  and  performance  for  the  demonstration  of  the  light  detection  and 
ranging  (LiDAR)  and  orthophotography  high  airborne  sensor  technologies  demonstrated  at 
Pueblo  Precision  Bombing  Range  (PBR)  #2  in  Otero  County,  Colorado,  and  at  Borrego  Military 
Wash  in  southern  California.  EiDAR  data  are  critical  to  the  overall  WAA  process  in  the  creation 
of  an  accurate  high-resolution  bare  earth  digital  elevation  model  (DEM)  for  ortho-correction  of 
all  other  remote-sensing  datasets.  EiDAR  and  orthophotography  are  both  valuable  for  the 
identification  and  delineation  of  possible  surface  munitions-related  features  such  as  target  circles 
and  craters  associated  with  munitions  use  at  the  site  and  for  development  of  base  mapping  layers 
for  site  visualization,  planning,  and  analysis. 

The  objectives  of  WAA  include  the  rapid  and  efficient  identification  of  areas  of  concentrated 
munitions  use  through  the  application  of  site  characterization  technologies.  Information  provided 
in  historical  records  forms  the  basis  for  initial  site  assessments;  field  surveys  employing  a  suite 
of  technologies  and  processing  techniques  develops  the  knowledge  of  these  sites  and  can  provide 
information  needed  to  support  decisions  at  various  stages  of  the  munitions  response  process. 
Equally  as  important  as  defining  areas  of  munitions  contamination  is  the  definition  of  areas  with 
no  indication  of  munitions  contamination.  The  evidence  derived  from  these  data  analyses  assists 
in  prioritizing  remediation  activities  by  providing  a  means  for  assessing  the  level  of  confidence 
in  conclusions  about  munitions  contamination  at  a  site. 

1.2  OBJECTIVES  OF  THE  DEMONSTRATION 

The  LiDAR  and  orthophotography  demonstrations  were  conducted  to  determine  the  utility  of 
these  datasets  to  identify  munitions-related  features,  including  target  areas,  craters,  and  range 
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infrastructure  features;  to  determine  the  aecuraey  of  the  datasets;  and  to  aequire  the  datasets  for 
site  eharaeterization  and  planning.  For  the  Pueblo  PBR  #2  site,  speeific  objeetives  ineluded 
eonfirmation  and  delineation  of  the  approximate  boundaries  of  two  doeumented  bombing  targets 
(Bombing  Targets  #3  and  #4  [BT3  and  BT4])  and  a  suspected  75-mm  air-to-ground  target  area 
(U.S.  Army  Corps  of  Engineers  [USAGE],  1995).  At  the  Borrego  Military  Wash  demonstration 
area,  the  speeifie  objeetives  ineluded  eonfirmation  and  delineation  of  the  two  targets  identified  in 
the  Arehive  Seareh  Report  (ASR)  (USAGE,  1997)  and  determination  of  whether  evidenee  exists 
of  any  previously  unknown  target  areas. 

1.3  REGULATORY  DRIVERS 

USAGE  is  the  lead  federal  ageney  under  the  Eormerly  Used  Defense  Site  (EUDS)  program. 
USAGE  administers  the  EUDS  Military  Munitions  Response  Program  (MMRP)  using  DoD 
investigation/cleanup  methods  based  on  the  U.S.  Environmental  Protection  Agency  (USEPA) 
Comprehensive  Environmental,  Response,  Compensation,  and  Eiability  Act  (CERCEA)  process. 

1.4  STAKEHOLDER/END  USER 

ESTCP  managed  the  stakeholder  issues  as  part  of  its  WAA  pilot  program.  ESTCP  used  a  proeess 
to  ensure  that  the  information  generated  by  the  high  airborne,  helieopter,  airborne,  ground 
validation  surveys  was  useful  to  a  broad  stakeholder  eommunity  (e.g.,  teehnieal  projeet  managers 
and  federal,  state,  and  loeal  governments,  as  well  as  other  stakeholders). 

1.5  DEMONSTRATION  RESULTS 

Data  eollection  for  the  Pueblo  PBR  #2  demonstration  took  plaee  in  two  phases:  Phase  I  in  2004, 
whieh  eovered  2,224  aeres  within  the  overall  7,500-aere  WAA  site,  and  Phase  II  in  2005,  whieh 
eovered  the  remaining  5,276  aeres  of  the  study  area.  Eor  both  Phases  I  and  II  data  eolleetions,  the 
expeeted  spatial  aeeuraeies  were  aehieved  for  both  the  LiDAR  and  orthophotography  datasets. 
Upon  analysis  of  the  proeessed  datasets,  targets,  eraters,  and  range  infrastructure  features  were 
deteeted.  These  results  aehieved  the  performanee  objeetives  outlined  in  the  demonstration  plan 
(Sky  Research,  2005a)  and  are  doeumented  in  the  final  report  (Sky  Researeh,  2008). 

Data  colleetion  for  the  demonstration  at  the  Borrego  demonstration  site  took  place  in  August  of 
2005.  The  expeeted  spatial  aeeuraeies  were  comprised  due  to  site  restrietions  on  the 
emplaeement  of  ground  fidueials.  Upon  analysis  of  the  proeessed  datasets,  targets  and  range 
infrastrueture  features  were  deteeted  and  the  results  eompared  to  ground  survey  data  eolleeted  by 
USAGE  for  validation.  The  final  report  doeuments  the  demonstration,  analysis,  and  results  (Sky 
Researeh,  2007a).  Although  the  performance  objeetives  were  not  all  met  for  the  demonstration, 
the  use  of  EiDAR  and  orthophotography  teehnologies  did  eonfirm  the  presenee  of  features 
requiring  further  investigation  and  geoloeated  these  features  more  aecurately  than  had  previously 
been  doeumented.  The  demonstration  showed  that  erosion  and  deposition  ean  impact  the 
persistenee  of  deteetable  features  over  time;  therefore,  the  potential  effeet  of  elimaetie  eonditions 
should  be  taken  into  aeeount  in  assessing  site  suitability  for  EiDAR  and  orthophotography. 

Table  1  summarizes  the  demonstrations  and  the  features  deteeted  in  the  orthophotography  and 
EiDAR  datasets. 
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Table  1.  LiDAR  and  Orthophotography  WAA  Demonstration  Summaries. 


Data  Collection 

Dates 

Acres 

Surveyed 

Features  Detected 

Pueblo  PBR  #2  Phase  I 

August  2004 

2,224 

Bomb  target  aiming  circle,  craters,  range 
infrastructure  features 

Pueblo  PBR  #2  Phase  I 

August  2005 

5,276 

Bomb  target  aiming  circle,  ship  targets,  craters, 
range  infrastructure  features 

Borrego  Military  Wash 

August  2005 

7,940 

Bomb  target  aiming  circle,  range  infrastructure 
features 

Overall,  LiDAR  and  orthophotography  technologies  are  efficient,  cost-effective  tools  for  WAA. 
As  the  first  part  of  an  integrated  WAA  investigation,  the  collection  of  thousands  of  acres  of  data 
per  day  and  the  analysis  of  LiDAR  and  orthophotography  datasets  yields  results  that  can  be  used 
to  characterize  the  extent  of  contaminated  areas  and  delineate  “clear”  areas  of  large  sites.  The 
utilization  of  these  technologies  for  WAA  will  result  in  a  reduction  in  the  overall  costs  of 
remediation  by  decreasing  the  number  of  acres  requiring  more  extensive  munitions  response 
actions  and  focusing  the  extent  of  further  investigations,  ultimately  yielding  more  efficient  use  of 
the  DoD’s  limited  cleanup  resources. 
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2.0 


TECHNOLOGY  DESCRIPTION 


2.1  TECHNOLOGY  DEVELOPMENT  AND  APPLICATION 

Airborne  sensors  utilized  for  this  demonstration  are  based  on  existing,  well-developed  airborne 
remote  sensing  technologies.  LiDAR  is  a  scanning  and  ranging  laser  system  capable  of 
producing  precise,  high-resolution  topographic  data.  The  technology  has  been  in  existence  for 
more  than  20  years,  although  the  commercial  application  for  topographic  maps  has  developed 
only  within  the  last  7  to  10  years.  Orthophoto  imagery  was  first  produced  more  than  over  30 
years  ago  and  was  first  created  using  a  combination  of  optical  and  digital  processing.  However, 
in  the  early  1990s  digital  orthophotography  became  a  commercial  production  reality  due 
primarily  to  the  increased  availability  of  more  powerful  computers  at  more  affordable  prices. 

The  application  of  these  technologies  to  characterization  of  features  related  to  munitions  and 
MEC  contamination  is  relatively  recent.  DoD  has  invested  significant  resources  demonstrating 
the  utility  of  applying  WAA  technologies,  including  LiDAR  and  orthophotography,  for  the 
characterization  of  MEC  contamination  on  large  DoD  sites,  including  the  WAA  pilot  program 
demonstrations.  In  addition,  production  level  surveys  have  been  conducted  on  behalf  of  the 
USAGE  at  a  number  of  former  and  active  sites. 

2.2  TECHNOLOGY  DESCRIPTION 

The  technologies  utilized  for  these  demonstrations  included  a  fixed-wing  plane  platform  housing 
the  data  acquisition  technologies  and  a  suite  of  data  processing  and  analysis  software.  The 
ALTM  3100  LiDAR  system,  ALTM  4K02  Digital  Metric  Camera,  and  the  Position  and 
Orientation  System  (POS)  were  mounted  in  the  Sky  Research  Cessna  208  (C208)  Caravan 
aircraft  (Eigure  1).  The  C208  is  an  unpressurized,  single-engine,  high  wing  turboprop  with  fixed 
landing  gear.  A  removable  composite  cargo  pod  provides  housing  for  the  equipment  and 
sensors;  both  the  LiDAR  and  orthophotography  sensors  were  installed  to  allow  for  concurrent 
data  collection. 


Figure  I.  The  C208  Fixed-Wing  Platform  Houses  the  Orthophotography  and  LiDAR 

Sensors  for  Concurrent  Data  Collection. 

2.2.1  LiDAR  Technology 

The  LiDAR  data  collection  system  used  for  these  demonstrations— an  Optech  ALTM  3100  laser 
scanner,  global  positioning  system  (GPS),  and  inertial  measurement  unit  (IMU)— is  capable  of 
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producing  precise,  high-resolution  topographie  data.  The  Optech  ALTM  3100  LiDAR  sensor 
speeifioations  are  summarized  in  Table  2. 


Table  2.  LiDAR  Specifications. 


Detector  type 

Optech®  LiDAR  ALTM  3100 

Spacing 

30  cm  to  5  m  spot  spacing 

Contour  interval 

Dependent  on  spot  spacing  with  an  approximate  1-m  (spacing)  to  30.5-cm 
(contour  interval)  ratio 

Operating  altitude 

80-3,500  m  above  ground  level  (AGL)  nominal 

Elevation  accuracy 

<15  cm  at  1,200  m;  1  sigma 
<25  cm  at  2,000  m;  1  sigma 
<35  cm  at  3,000  m;  1  sigma 

Horizontal  accuracy 

Better  than  1/3,000  x  altitude;  1  sigma 

Range  accuracy 

2-3  cm,  single  shot 

Range  resolution 

1  cm 

Measurement  rate 

33,000  to  100,000  measurements  per  sec 

Scan  angle 

0  to  ±  25° 

Swath  width 

Variable  from  0  to  0.93  x  altitude 

Scan  frequency 

0-70  Hz,  depending  on  scan  angle 

Laser  classification 

Class  IV  (FDA  Code  of  Federal  Regulations  [CFR]  21) 

Laser  repetition  rate 

33  kHz  (max.  altitude  AGL  3,500  m) 

50  kHz  (max.  altitude  AGL  2,500  m) 

70  kHz  (max.  altitude  AGL  1,700  m) 

100  kHz  (max.  altitude  AGL  1,100  m) 

Operating  temperature 

10-35°  C 

Humidity 

0-95%  noncondensing 

The  laser  seanner  operates  by  emitting  high-frequeney  infrared  laser  beams.  The  seanner  reeords 
the  time  differenee  between  the  emission  of  the  laser  pulses  and  the  reception  of  the  refleeted 
signal.  A  mirror  mounted  in  front  of  the  laser  rotates,  direeting  the  laser  pulses  to  sweep  baek 
and  forth  perpendieular  to  the  flight  direetion,  whieh  allows  the  laser  seanner  to  eolleet  swaths  of 
topographie  data  as  the  aireraft  moves  forward.  The  position  of  the  aireraft  is  determined  by 
proeessing  differential,  dual-frequeney,  kinematic  GPS  observations.  The  GPS  located  in  the 
aireraft  is  supported  by  several  ground  stations  that  are  loeated  within  the  vicinity  of  the 
aequisition  area.  The  IMU  determines  the  orientation  of  the  aireraft  (piteh,  roll,  and  yaw)  during 
data  eolleetion.  By  eombining  the  IMU  data  with  the  post-proeessed  GPS  data,  the  exaet 
trajeetory  of  each  laser  pulse  is  determined  during  data  proeessing. 

During  data  aequisition  flights,  the  sensor  operator  observes  the  real-time  LiDAR  swath 
coverage  to  assure  full  eoverage  of  the  survey  area.  The  operator  also  monitors  in-flight  position 
dilution  of  preeision  (PDOP)  and  GPS  satellite  coverage.  If  tolerance  thresholds  of  either  are 
exeeeded,  data  aequisition  activities  are  put  on  hold  until  aeeeptable  eonditions  resume.  After 
the  data  aequisition  flights,  data  from  GPS  base  stations  are  eheeked  against  in-flight  GPS  data 
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for  concurrence.  Once  data  quality  assessments  are  completed,  all  data  (image  and  aneillary)  are 
transferred  to  a  eentralized  location  for  preprocessing  and  quality  control  analysis. 

Processing  of  the  raw  LiDAR  datasets  employs  a  variety  of  software  technologies.  Sky  Research 
uses  the  following  technologies: 

•  POSPac/POSGPS®  software  for  GPS  data  processing 

•  POSProe  software  for  combining  postprocessed  GPS  data  with  IMU  data 

•  Optech’s  REALM  software  for  initial  proeessing  and  output  of  LiDAR  point 
eloud  data 

•  Terrasolid’s  TerraScan  software  for  classification  of  LiDAR  points  into 
vegetation,  ground,  and  “other,”  creating  bare  earth  and  surface  model  digital 
terrain  models  (DTM) 

•  Environmental  Systems  Researeh  Institute’s  (ESRI)  AreGIS  geoprocessing  seripts 
and  Natural  Neighbor  interpolation  for  interpolating  both  DTM  models  to  DEMs 
and  shaded  relief  imagery 

•  Visual  Learning  Systems’  Leature  Analyst  AreGIS  extension  and  eustom 
interactive  data  language  (IDL)  software  algorithms  to  locate,  detect,  and 
characterize  microtopographic  features,  including  craters. 

2,2.2  Orthophotography  Technology 

Lor  these  demonstrations,  a  high-resolution  Optech  ALTM  4K02  digital  metric  camera  with 
high-resolution  eharged  couple  device  (CCD)  backing  was  mounted  in  the  aireraft  to  eapture  the 
aerial  photography.  The  system  works  as  follows:  the  CCD  eonverts  light  into  eleetrons,  whieh 
are  enumerated  and  converted  into  a  digital  value.  The  ability  of  a  CCD  to  aeeurately  measure 
and  convert  the  value  of  eleetrons  into  digital  format  is  the  measure  of  quality.  As  a  small  format 
system,  the  ALTM4K02  camera  offers  a  36°  field  of  view  (LOV),  minimizing  layback  distortion 
at  the  edges  of  images  and  minimal  image  distortion  during  the  orthoreetification  phase  of 
proeessing.  The  manufacturer’s  specifications  for  the  eamera  used  for  data  colleetion  are 
summarized  in  Table  3.  The  camera  is  linked  to  a  computer  that  controls  the  frequency  and 
length  of  exposures,  resulting  in  overlapping  images.  Information  collected  from  the  GPS  and 
IMU  are  used  to  reetify  the  aerial  photographs.  This  is  aecomplished  by  assigning  a  geographic 
coordinate  to  eaeh  image  derived  from  the  processing  of  the  GPS  data.  In  addition,  distortions 
ereated  by  camera  tilt,  lens  distortion,  and  terrain  displaeement  are  removed  to  produce  an 
orthophotograph. 
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Table  3,  Camera  Specifications. 


Detector  type 

Optech  ALTM  4K02  Digital  Metric  Camera  DSS  301  SN0046- 
55-mm  lens 

Lens  type 

Zeiss  Distagon 

Focal  length 

55.073  mm 

Field  of  view 

36° 

CCD  specifications 

4,092  (along  flight)  x  4,079  (cross  flight) 

Pixel  size  of  0.000138  in 

Shutter  speed 

1/125  to  1/4000  sec 

Principal  point 

Xppac  (mm)  -0.390,  Accuracy  0.0036 

Yppac  0.222,  Accuracy  0.0036 

Measured  from  image  center  (pixel  size  =  9  microns) 

Pixel  non-squareness 

1.0,  Accuracy  0.0000001 

VIS  calibrated  gain  value 

0.98 

VIS  calibrated  ISO 

300 

VIS  calibrated  exposure  compensation 

-0.70 

During  the  data  acquisition  flights,  the  sensor  operator  observes  the  real-time  photograph 
footprint  coverage  to  assure  required  percentage  of  overlap  for  the  survey  area.  The  operator 
observes  real-time  photo  display  for  verification  of  image  quality.  The  operator  also  monitors 
in-flight  PDOP  and  GPS  satellite  coverage.  If  tolerance  thresholds  of  either  are  exceeded,  data 
acquisition  activities  are  put  on  hold  until  acceptable  conditions  resume.  After  the  data 
acquisition  flights,  data  from  GPS  base  stations  are  checked  against  in-flight  GPS  data  for 
concurrence.  Once  data  quality  assessments  are  completed,  all  data  (image  and  ancillary)  are 
transferred  to  a  centralized  location  for  preprocessing  and  quality  control. 

Processing  the  raw  data  sets  employs  a  variety  of  software  technologies.  Sky  Research  uses  the 
following  technologies: 

•  POSPac/POSGPS®  software  for  GPS  data  processing 

•  POSProc  software  for  combining  postprocessed  GPS  data  with  IMU  data 

•  Raw  photographs  developed  into  TIFF  format  with  manufacturer-calibrated  true- 
color  (VIS)  filter 

•  DSS  MissionView  2.0  for  downloading  images  from  removable  drives 

•  POSEO  4. 1  for  processing  the  photographs  to  sync  with  GPS  data 

•  ZI  Imaging  Image  Station  Auto  Triangulation  (ISAT)  software  to  combine 
formatted  image  files  with  exterior  orientation  files 

•  ImageStation  OrthoPro  software  for  rectification  of  the  photography  to  the  DTM. 
2,2,3  Data  Analysis 

Once  processed,  the  LiDAR/orthophotography  data  sets  are  analyzed  to  characterize  any 
munitions-related  features  that  are  visible  in  the  data  sets  and  that  may  be  useful  in 
characterizing  munitions  contamination  present  at  the  site.  These  surface  features  may  include 
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high  explosive  (HE)  eraters,  target  and  range  berms,  burial  trenehes,  abandoned  serviee  roads, 
artillery  targets,  and  other  features  where  a  surfaee  topographie  or  soiEvegetation  expression  is 
observed  in  the  LiDAR  and/or  orthophotography  data  sets.  Extraetion  of  the  potential  munitions- 
related  features  from  the  orthophotography  and  EiDAR  data  sets  is  both  an  automated  and 
analyst-performed  task  that  eombines  multiple-overlay  image  interpretation  with  automated 
spatial  feature  reeognition  proeesses  utilizing  AreGIS  and  Visual  Eearning  Systems  software. 

2.3  PREVIOUS  TESTING  OF  THE  TECHNOLOGY 

Component  WAA  teehnologies  have  been  developed  and  tested  at  a  number  of  defense  sites  over 
the  past  ten  years.  A  WAA  at  Former  Eowry  Bombing  and  Gunnery  Range  (ELBGR)  in 
Colorado  was  the  first  praetieal  applieation  of  the  use  of  EiDAR  and  orthophotography 
methodology  for  MEC  site  assessment.  However,  at  the  time  the  EEBGR  WAA  was  eondueted, 
mueh  of  the  site  had  been  surfaee-eleared  of  munitions  eontamination  at  known  sites, 
signifieantly  eomplieating  the  analysis.  Sinee  then,  demonstrations  of  EiDAR/orthophotography 
teehnologies  have  been  eondueted  at  eaeh  WAA  pilot  program  demonstration  site. 

2.4  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 

As  with  all  eharaeterization  teehnologies,  site-speeifie  advantages  and  disadvantages  exist  that 
dietate  the  level  of  sueeess  of  their  applieation.  However,  in  general,  the  advantages  of  high 
airborne  sensor  WAA  teehnologies  inelude: 

•  the  ability  to  eharaeterize  very  large  areas; 

•  eharaeterization  of  the  site  in  terms  of  munitions-related  features,  whieh  provides 
a  more  robust  strueture  to  the  overall  eoneeptual  site  model  (CSM); 

•  ability  to  deploy  multiple  sensors;  and 

•  aneillary  produets,  ineluding  high  fidelity  DEMs  and  high  resolution  photography 
that  ean  be  used  for  a  wide  variety  of  site  aetivities. 

Eimitations  of  the  demonstrated  WAA  teehnologies  inelude: 

•  Use  of  high  airborne  sensors  is  not  intended  to  deteet  individual  munitions. 

•  Site  physiography,  sueh  as  terrain  and  vegetation,  ean  eonstrain  the  use  of 
teehnology  for  munitions-related  feature  deteetion. 

•  EiDAR  and  orthophotography  teehnologies  ean  only  deteet  features  with 
expression  on  the  earth  surfaee. 

•  Eaeh  teehnology  has  survey  rate  and  eost  versus  deteetion  fidelity  trade-offs. 
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3.0  DEMONSTRATION  DESIGN 


3.1  PERFORMANCE  OBJECTIVES 

Performance  objectives  are  critical  demonstration  components  because  they  provide  the  basis  for 
evaluating  the  technology  performance.  For  these  demonstrations,  primary  and  secondary 
performance  objectives  were  established  and  documented  (Table  4).  These  objectives  include  the 
georeference  accuracy  of  the  datasets  and  the  ability  to  detect  surface  features  indicative  of 
potential  MFC  contamination,  including  target  areas,  craters,  and  range  infrastructure. 

3.2  TEST  SITE  SELECTION 

Pueblo  PBR  #2  was  initially  selected  for  demonstration  in  2004  based  on  the  site  characteristics, 
which  were  expected  to  be  amenable  to  feature  detection  in  high  airborne  remote  sensing  data 
sets.  In  2005,  ESTCP  created  the  WAA  pilot  program  in  response  to  the  DSB  Task  Force  report 
and  congressional  interest,  to  validate  the  application  of  a  number  of  recently  developed 
technologies  as  a  comprehensive  approach  to  WAA.  The  WAA  pilot  program  demonstration 
sites  were  selected  based  on  criteria  selected  by  the  ESTCP  Office  in  coordination  with  the 
WAA  Advisory  Group.  In  response  to  that  selection,  the  demonstration  area  for  Pueblo  PBR  #2 
was  expanded  and  a  second  data  collection  (Phase  II)  was  conducted  in  2005  to  encompass  a 
second  documented  bombing  target  and  a  suspected  bombing  target.  Borrego  Military  Wash  was 
also  selected  as  a  demonstration  site  at  that  time. 

3.3  TEST  SITE  HISTORY/CHARACTERISTICS 

Pueblo  PBR  #2  was  used  as  a  World  War  Il-era  military  training  facility,  located  in  the  southern 
part  of  Otero  County,  Colorado.  Within  the  105 -square-mile  (67,770  acres)  FUDS,  the 
demonstration  area  consists  of  approximately  7,500  acres  encompassing  the  two  bombing  targets 
(BT3  and  BT4)  and  the  suspected  75-mm  air-to-ground  target  area  documented  in  the  ASR 
(USAGE,  1995)  and  shown  in  Figure  2.  It  was  postulated  prior  to  the  demonstration  that 
undocumented  target  locations  could  potentially  lie  within  the  study  area.  Eand  within  the  study 
area  is  primarily  in  federal  ownership  managed  by  the  U.S.  Forest  Service  as  the  Comanche 
National  Grasslands  with  portions  leased  to  private  owners  or  owned  by  the  State  of  Colorado. 
Somewhat  less  than  2,000  acres  of  the  7,500-acre  study  area  are  privately  owned,  nonresidential 
gazing  lands. 

The  Borrego  Military  Wash  demonstration  site  was  a  7,500-acre  subsite  of  the  Borrego 
Maneuver  Area  and  is  representative  of  a  large  number  of  munitions  sites  associated  with  the 
nearby  California- Arizona  Maneuver  Area.  The  Borrego  Maneuver  Area  encompasses  desert, 
mountains,  and  badlands,  with  negligible  amounts  of  arid  climate  vegetation  species.  It  was 
acquired  by  use  permit  from  the  State  of  California  in  1942  for  use  as  a  force-on-force  maneuver 
area.  The  Army  closed  their  portion  in  1944  but  the  Navy  continued  to  use  their  areas  through 
1953.  Training  activities  at  the  Borrego  Maneuver  Area  included  preparing  combat  troops  for 
desert  warfare,  instruction  of  mechanized  artillery  service  units  and  staff,  antiaircraft  training, 
and  practice  bombing. 
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Table  4.  Primary  and  Secondary  Performance  Objectives, 


Type  of 
Performance 
Objective 

Primary  Performance 
Criteria 

Expected  Performance 
(Metric) 

Performance 
Confirmation  Method 

Actual  Performance 
Metric  Met? 

Qualitative 

(Primary) 

Ease  of  use  and  effieieney 
of  operations 

Effieieney  and  ease  of  use  meets  design 
speeifieations 

General  observations 

Pueblo  PER  #2:  Met 

Borrego:  Met 

Quantitative 

(Primary) 

Georeferenee  position 
aeeuraey  for  eaeh  sensor 
system 

LiDAR: 

Vertieal  aeeuraey  -  15-em  root  mean 
square  error  (RMSE); 

Horizontal  aeeuraey  -  40-em  RMSE 

Pueblo  PER  #2: 

Comparison  of  data  sets 
with  ground  fidueials 

Borrego:  Caleulated  from 
average  offset  for  eight 
eomers  of  two  rake 
stations* 

Pueblo  PBR  #2:  Met 

Borrego:  Not  Met* 

Orthophotography : 

Horizontal  aeeuraey  -  40-em  RMSE 

Pueblo  PBR  #2:  Met 

Borrego:  Not  Met* 

Target  area  deteetion 

>0.90  of  target  areas  having 
topographie  aiming  point  features 

Comparison  of  ortho  and 
LiDAR  data  analysis 
results  with  ground 
validation  data** 

Pueblo  PBR  #2:  Met 

Borrego:  Not  Met*** 

Quantitative 

(Secondary) 

Crater  deteetion 

>0.75  (eraters  <1  m) 

>0.90  (eraters  >  1  m) 

Comparison  of  LiDAR  data 
analysis  results  with 
ground  validation  data** 

Pueblo  PBR  #2:  <1  m:  Not 
within  deteetion  limits; 

>1  m:  Performanee  met 
Borrego:  No  eraters  deteeted 
or  loeated  during  ground 
survey 

Range  infrastmeture 
deteetion 

>0.90 

Comparison  of  ortho  and 
LiDAR  data  analysis 
results  with  ground 
validation  data** 

Pueblo  PBR  #2:  N/A 

Borrego:  Identified  features 
verified  as  range 
infrastmeture 

*  Performance  confirmation  method  reported  is  typically  comparison  of  datasets  with  ground  fiducials;  however,  for  the  Borrego  demonstration,  another  confirmation  method  (as  described  in  Section 


4. 1 .2.2)  was  used  because  of  restriction  on  emplacements  of  fiducials,  which  also  resulted  in  a  lower  accuracy  than  anticipated. 

**  For  the  Borrego  demonstration,  visual  site  inspection  results  were  utilized  for  validation  purposes. 

***  For  the  Borrego  demonstration,  five  target  features  were  detected  and  one  target  feature  not  detected  resulting  in  a  calculated  performance  value  of  0.83. 
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Figure  2,  The  WAA  Demonstration  Area  (in  Yellow)  is  Located  Within  the  Former 
Puehlo  PER  #2  in  Otero  County,  Colorado, 


The  Borrego  Military  Wash  eontains  two  sub-areas  identified  in  the  ASR.  The  area  designated  as 
E-1  eontains  a  variety  of  targets  and  munitions-related  features,  ineluding  eontained  bombing, 
strafing,  and  roeket  targets  with  rake  (observation)  stations  that  are  firing  points  for  40-mm  and 
90-mm  antiaircraft  weapons  systems.  There  also  may  have  been  an  air-to-ground  railway  strafing 
target  as  well  as  a  bermed  area  for  ground-to-ground  firing  as  part  of  an  Army  anti-mechanized 
target.  Sub-area  E-2  was  designated  as  a  safety  buffer  area  around  Sub-area  E-1.  The  Borrego 
Military  Wash  7,940-acre  demonstration  area  was  buffered  substantially  (by  0.5  km)  to  ensure 
that  all  the  extents  of  the  project  area  contained  an  adequate  number  of  tie  points  for  the 
orthophotography  dataset.  Eigure  3  shows  the  Borrego  Maneuver  Area  boundaries,  Borrego 
Military  Wash  boundaries,  and  the  E-1  and  E-2  Sub-area  boundaries. 

3.4  PHYSICAL  SETUP  AND  OPERATION 

3.4.1  LiDAR  Calibration 

Prior  to  data  collection  flights,  a  calibration  flight  was  completed  over  a  known  calibration  site 
located  at  the  Sky  Research  facility  in  Ashland,  Oregon.  This  calibration  flight  served  several 
purposes— to  assess  the  alignment  and  offsets  between  the  scanning  mirror  of  the  sensor,  the 
IMU,  and  the  GPS  antenna  on  the  aircraft  and  to  compare  the  results  of  the  flight  with  known 
survey  points. 

3.4.2  Mobilization/Demobilization 

Mobilization  and  demobilization  for  these  demonstrations  required  transport  of  the  plane, 
equipment,  pilot,  and  sensor  operators  from  Ashland,  Oregon,  to  the  base  of  operations.  Ground 
personnel  deployed  from  Denver,  Colorado,  to  establish  ground  fiducials,  establish  and  operate 
GPS  base  stations,  and  provide  logistical  support  returned  to  Denver  at  the  conclusion  of  the 
demonstrations. 

3.4.3  Crater  Emplacement 

Ten  craters  were  manually  created  at  the  Pueblo  demonstration  site  for  an  assessment  of  crater 
detection  ability  using  EiDAR  data.  The  emplaced  craters  consisted  of  a  controlled  series  of 
circular  depressions  and  eight  existing  HE  craters.  The  ability  to  detect  each  crater  using  the 
EiDAR  imagery  was  classified  into  one  of  four  categories— high,  medium  low,  and  no  detect 
(ND). 

3.4.4  Ground  Control 

A  ground  support  team  operated  GPS  base  stations,  collected  GPS  road  calibration  transects, 
placed  ground  control  target  panels  for  the  EiDAR  and  orthophotography  data  collection,  and 
collected  GPS  data  during  the  data  collection  flights. 

3.4.5  Navigation  Systems 

An  Applanix  510  AW  POS  system  was  co-mounted  with  both  sensors  to  record  the  aircraft’s 
GPS  position  utilizing  a  dual  frequency  GPS  receiver  and  attitude  (pitch,  roll,  and  yaw)  utilizing 
an  IMU.  The  Optech  AETM-NAV  software  package  was  used  for  flight  navigation  and  allowed 
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Figure  3,  The  WAA  Demonstration  Was  Conducted  in  Sub-Areas  E-1  and  E-2  of  the  Borrego  Military  Wash 
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the  sensor  operator  to  view  in  real  time  the  swath  of  the  laser  system,  images,  PDOP  levels,  and 
number  of  satellites,  as  well  as  any  problems  with  the  laser  or  eamera  system. 

3,4.6  Period  of  Operation 

The  Phase  I  data  eolleetion  flights  at  Pueblo  PBR  #2  oeeurred  on  August  20  and  23,  2004.  A 
total  of  5  hours  of  flight  time  was  needed  to  eolleet  data  over  approximately  6,700  aeres.  Phase  II 
data  eolleetion  oeeurred  on  August  6,  2005,  in  one  3.5-hour  flight  over  approximately  6,600 
aeres.  The  Borrego  data  eolleetion  oeeurred  on  August  16,  2005.  One  flight  lasting  3.6  hours  was 
needed  to  eolleet  data  over  approximately  8,900  aeres  (note,  the  8,900  acres  of  survey  data 
includes  substantial  buffering  to  ensure  sufficient  data  collection). 

3.5  OPERATING  PARAMETERS  FOR  THE  TECHNOLOGY 

The  flight  parameters  for  the  data  acquisition  surveys  were  set  during  the  planning  stages  to  meet 
the  required  accuracies  for  the  LiDAR  and  orthophotography  data  sets  at  flight  altitudes  800  m 
of  AGL  and  flight  speeds  of  54  m/sec  (105  knots).  Flight-line  spacings  were  approximately  230 
m  to  allow  50%  overlaps  with  the  560-m  swath  widths  achievable  at  800  m  for  LiDAR  data 
acquisition.  Average  elevation  sample  postings  were  planned  for  less  than  40  cm,  with  planned 
vertical  accuracy  of  15  cm  and  horizontal  accuracy  of  40-cm  RMSE.  At  the  planned  collection 
parameters,  vertical  accuracies  relative  to  adjacent  sample  points  were  predicted  to  be  better  than 
5-cm  RMSE,  providing  very  high  resolution  surface  modeling  capabilities.  Orthophotography 
data  were  collected  concurrently  with  the  EiDAR  data  collection  and  collected  on  every  other 
EiDAR  flight  line,  with  cross-track  overlaps  of  36%  and  along-track  overlaps  of  30%.  Tables  5 
and  6  summarize  the  operating  parameters  for  the  data  collections. 

Table  5,  Acquisition  Parameters  for  LiDAR  Data  Collection. 


Measurements  per  second 

100,000 

Scan  width 

370  m 

Scan  overlap 

50%  (185  m) 

Scan  frequency 

60  Hz 

Scan  angle 

+/-  13 

Spot  spacing 

0.44  m 

Table  6.  Acquisition  Parameters  for  Color  Orthophotography  Data  Collection, 


Field  of  view 

36° 

Imaging  swath/photo  coverage 

36%  cross-track  overlap;  30%  along-track 
overlap 

Ortho-rectified  pixel  size,  resampled 

0.16  m/pixel 

3.6  ANALYTICAL  PROCEDURES 

LiDAR  processing  transforms  raw  binary  data  into  a  functional  DEM.  Primary  processing  of  raw 
digital  camera  data  results  in  a  seamless,  orthometrically  correct  24-bit  red-green-blue  (RGB) 
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aerial  photomap  of  the  site.  Seeondary  analyses  ineluded  eonversion  of  proeessed  sensor  datasets 
into  suspeeted  munitions-related  feature  datasets  utilizing  a  variety  of  proeessing  and  analytieal 
teehniques.  Several  image  proeessing  steps  were  used  to  generate  derivative  raster  datasets  that 
enhanee  feature  detection,  followed  by  a  systematic  manual  inspection  and  interpretation  of  the 
imagery  in  a  geographic  information  systems  (GIS)  workstation  environment  that  facilitates 
multiple  image  overlays  with  transparency  and  edge-sweep  controls,  contrast  stretching, 
multiple-band  blending  in  an  RGB  color  model,  and  other  visualization  tools.  Interpreted  image 
features  were  incorporated  into  the  site  geodatabase  as  vector  point  and  polygon  map  features. 

3.6.1  Computation  of  Derivative  LiDAR  Images  and  Analysis  Grids 

The  LiDAR  bare  earth  DEM  was  used  to  compute  two  different  derivative  images,  including  a 
“hillshade”  image  and  an  “analytic”  image.  The  hillshade  was  computed  using  a  raster  analysis 
function  that  computes  the  hypothetical  illumination  of  a  surface  by  determining  illumination 
values  on  a  cell-by-cell  basis  for  each  cell  in  the  image.  The  analytic  image  was  a  high-pass  filter 
of  the  bare  earth  DEM  computed  by  subtracting  the  elevation  value  of  each  DEM  cell  from  the 
average  of  the  surrounding  cells  in  a  defined  circular  neighborhood.  This  process  resulted  in  an 
image  that  emphasizes  microtopographic  features  in  the  image  of  a  scale  correlated  with  the 
filter’s  search  radius.  To  enable  an  efficient  and  systematic  analysis,  the  study  area  was 
subdivided  into  100-m  grid  cells  and  further  subdivided  into  20-m  cells.  This  two-tiered  grid 
system  was  used  by  the  analyst  to  track  progress  and  ensure  complete  and  even  review  of  the 
imagery. 

3.6.2  Target  Feature  Identification  and  Extraction 

Datasets  within  each  grid  cell  were  systematically  examined  in  ArcGIS  for  target  features, 
including  circles,  “cross-hair”  aiming  points,  ship  outlines,  and  rectangular  shapes  that  could 
represent  airstrips,  buildings,  or  other  target  types.  As  features  were  identified  in  an  image  and 
corroborated  in  other  imagery,  outlines  were  digitized  by  the  operator  into  a  feature-class 
geodatabase  layer  and  assigned  attributes  such  as  area,  centroid,  primary  and  corroborating 
sensors,  description,  and  feature  type. 

3.6.3  Crater  Feature  Identification  and  Extraction 

Crater  feature  identification  and  extraction  utilized  the  EiDAR-derived  hillshade  and  analytic 
high-pass  DEM  images.  Eor  purposes  of  a  crater  detection  analysis,  HE  impact  craters  were 
defined  as  circular  or  semicircular  depressions  of  any  size  up  to  20  m  in  diameter.  Perimeter 
circularity,  concave  bottom  profile  and  raised  rim  were  considered  to  be  diagnostic  attributes.  To 
classify  a  depression  as  a  crater,  any  significant  irregularity  of  shape  (departure  from  circularity) 
should  be  explained  by  adjacent  crater  features,  rock  outcrops,  or  recent  disturbance.  An 
automated  circular  depression  detection  algorithm  using  the  Eeature  Analyst  extension  for 
ArcGIS  combined  with  a  custom  radial  search  pattern  was  used  for  detecting  craters,  which  were 
later  manually  verified  or  rejected  using  the  EiDAR  hillshade  and  analytic  images.  Verified 
craters  were  digitized;  overlapping  craters  were  digitized  as  overlapping  circles.  After  the  craters 
were  digitized  for  each  analysis  grid,  the  operator  ran  a  custom  spatial  analysis  script  within  the 
ArcGIS  environment  that  extracted  the  high  and  low  elevations  within  each  circle  from  the 
underlying  DEM,  and  stored  these  elevations  and  the  difference  (crater  depth)  as  attributes  for 
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each.  Crater  area,  perimeter  length,  and  centroid  coordinates  were  also  saved  as  feature 
attributes. 

To  visualize  the  distribution  of  craters  across  the  study  area,  a  density  analysis  was  performed  by 
computing  a  kernel  density  raster  whose  cell  values  each  describe  the  crater  density  in  craters  per 
hectare  of  a  circular  neighborhood  around  each  cell.  Changes  in  the  neighborhood  radius  affect 
the  resulting  density  surface,  with  larger  radii  producing  a  more  generalized  density  model  and 
smaller  radii  producing  more  detail.  Therefore,  a  neighborhood  radius  appropriate  to  the  crater 
density  and  patterns  was  interactively  determined  to  produce  a  density  surface  most  appropriate 
to  the  distribution  of  craters  across  the  study  area  landscape. 

3,6,4  Range  Infrastructure  Identification  and  Extraction 

The  LiDAR  and  orthophoto  data  sets  were  analyzed  to  extract  other  anthropogenic  features  to  aid 
interpretation  and  characterization  of  the  MEC  contamination  patterns  by  the  spatial  correlation 
of  extracted  features  such  as  transport  routes  and  evidence  of  excavation  activities  with 
documentary  site  usage  information.  The  feature  extraction  methodology  used  to  extract 
infrastructure  features  was  essentially  the  same  as  that  described  for  the  identification  and 
extraction  of  target  features  and  was  conducted  concurrently  with  that  extraction. 
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4.0  PERFORMANCE  ASSESSMENT 


4.1  PERFORMANCE  DATA 

4.1.1  Crater  Detection  Analysis 

The  crater  detection  analysis  completed  as  part  of  the  Pueblo  PBR  #2  demonstration  showed  that 
LiDAR  imagery  can  detect  HE  craters  at  a  crater  diameter  threshold  of  about  150  cm  and  a 
diagnostic  crater  diameter  threshold  (i.e.,  terrain  depression  can  be  recognized  as  a  probable  HE 
crater)  of  about  350  cm.  Both  the  hillshade  and  analytic  high-pass  imagery  were  evaluated  for 
crater  detection;  the  analytic  image  was  found  to  be  more  useful  because  of  the  improved 
detectability  of  the  characteristic  raised  berm  of  excavated  soil  around  the  perimeter  of  the 
craters  and  the  improved  contrast  of  even  small  circular  depressions  with  surrounding  terrain. 
Table  7  shows  the  results  of  the  analysis  and  includes  the  detection  classification  (i.e.,  high, 
medium,  low,  ND)  for  both  the  EiDAR  derived  hillshade  and  analytic  high  pass  imagery  types 
and  the  estimated  diameter  and  depth. 

Table  7,  Diagnostic  Detectability  of  Crater  Features  in  LiDAR  Imagery, 


Crater 

ID 

Hillshade 

Analytic 

Diameter 

(cm) 

Depth 

(cm) 

17 

High 

High 

1121 

90 

15 

High 

High 

564 

50 

16 

High 

High 

534 

37 

19 

High 

High 

501 

36 

13 

High 

High 

399 

31 

14 

Medium 

High 

353 

24 

20 

Low 

Medium 

288 

14 

18 

Low 

Medium 

194 

12 

1 

Low 

Medium 

177 

30 

3 

Low 

Low 

166 

30 

7 

ND 

Low 

104 

28 

12 

ND 

ND 

74 

13 

2 

ND 

ND 

74 

11 

10 

ND 

ND 

65 

24 

5 

ND 

ND 

63 

25 

11 

ND 

ND 

54 

28 

9 

ND 

ND 

46 

20 

8 

ND 

ND 

45 

20 

4,1.2  Spatial  Accuracy 
4,1.2.1  Pueblo  PBR  #2 

The  EiDAR  and  orthophotography  spatial  accuracy  performance  goals  were  met  for  the  Pueblo 
PBR  #2  demonstrations.  To  determine  LiDAR  spatial  accuracy,  horizontal  offsets  were 
calculated  by  determining  the  centroid  of  the  modeled  perimeter  versus  surveyed  center  point. 
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Vertical  offsets  were  calculated  from  average  feature  surface  elevation  versus  surveyed  center 
point  elevation.  To  determine  orthophotography  spatial  accuracy,  image  coordinates  of  ground 
targets  were  compared  to  GPS  position,  and  X-Y  offsets  were  calculated.  Accuracy  results  are 
provided  in  Tables  8  and  9. 

Table  8,  LiDAR  Data  Accuracy  Results  at  Pueblo  PER  #2, 


Accuracy  Metrics 

Phase  I 

Data  (cm) 

Phase  II 
Data  (cm) 

YRMSE 

7.0 

9.2 

Y  linear  error  (95%  eonfidenee  level) 

13.7 

18.1 

XRMSE 

27.4 

7.6 

X  linear  error  (95%  eonfidenee  level) 

53.7 

15.2 

RMSE  horizontal  radial  error  (68.3%  eonfidenee  level) 

28.2 

8.5 

Horizontal  radial  error  (95%  eonfidenee  level) 

42.1 

16.6 

RMSE  vertieal  linear  error  (68.3%  eonfidenee  level) 

11.1 

3.2 

Vertieal  linear  error  (95%  eonfidenee  level) 

21.8 

6.3 

Table  9.  Ortbopbotograpby  Data  Accuracy  Results  at  Pueblo  PER  #2, 


Accuracy  Metrics 

Phase  I  Data 
(cm) 

Phase  II 
Data  (cm) 

YRMSE 

13.1 

9.9 

Y  linear  error  (95%  eonfidenee  level) 

25.7 

19.5 

XRMSE 

10.3 

12.0 

X  linear  error  (95%  eonfidenee  level) 

20.2 

23.5 

RMSE  horizontal  radial  error  (68.3%  eonfidenee  level) 

28.6 

26.8 

Horizontal  radial  error  (95%  eonfidenee  level) 

56.2 

52.6 

4,1.2.2  Eorrego 

For  the  Borrego  Military  Wash  demonstration,  the  LiDAR  and  orthophotography  spatial 
accuracy  performance  goals  were  not  met  due  to  the  restrictions  on  the  emplacement  of  aerial 
targets.  This  restriction  precluded  the  use  of  a  standardized  evaluation  method  for  determining 
spatial  accuracy  of  the  datasets;  instead,  the  georeference  accuracy  of  both  the  orthophoto  and 
LiDAR  datasets  depended  entirely  upon  the  airborne  GPS/POS  georeferenced  data.  The  rake 
stations  were  used  to  calculate  spatial  accuracy  as  the  eight  corners  of  the  two  rake  stations  were 
clearly  delineated  in  the  processed  datasets.  To  use  the  rake  stations  for  this  assessment,  the  rake 
station  locations  were  defined  in  the  field  by  real-time  kinematic  GPS  (RTK  GPS);  the  shape  of 
each  rake  station  feature  digitized  using  the  hillshade  LiDAR  image;  and  the  offset  between  the 
two  calculated.  The  measured  offsets  for  the  eight  comers  were  then  averaged  to  report  the 
estimated  accuracies.  It  should  be  noted  that  these  estimates  do  not  have  measures  of  variance 
that  provide  meaningful  information  about  the  confidence  error  of  the  estimate  since  they  are 
associated  with  a  very  limited  number  and  scope  of  measures.  This  methodology  was  used  to 
calculate  the  mean  horizontal  offset  (89  cm)  and  the  average  horizontal  offset  (22  cm)  in  the 
LiDAR  dataset  and  the  average  horizontal  spatial  offset  (92  cm)  of  the  orthophotography  dataset 
(Table  10). 
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Table  10,  LiDAR  and  Orthophotography  Data  Accuracy  Results  at  Borrego, 


Dataset 

Accuracy  Metrics 

Results 
(in  cm) 

LiDAR 

RMSE  vertical  linear  error  (68.3%  confidence  level) 

62 

LiDAR 

Vertical  linear  error  (95%  confidence  level) 

22 

LiDAR 

Horizontal  accuracy  estimate 

89 

Orthophotography 

Horizontal  accuracy  estimate 

92 

4,1,3  Target  Area  Detection 
4,1,3,1  Pueblo  PBR  #2 

Target  features  antieipated  to  be  present  based  on  historieal  data  included  target  circles  at 
bombing  BT3  and  BT4  observed  in  the  1951  aerial  photography.  The  1995  ASR  field 
inspections  located  the  aiming  circle  at  BT3  but  did  not  observe  other  target  features  (other 
potential  HE  craters).  The  1995  ASR  mentions  the  construction  of  an  air-to-ground  pattern 
gunnery  range,  conversion  of  five  of  the  nine  precision  bombing  targets  to  E-1  sonic  bomb 
scoring  targets,  and  the  construction  of  skip  bombing  (ship)  and  submarine  targets  several  years 
after  the  range  was  initially  put  into  operation.  Locations  for  these  upgraded  targets  were  not 
documented.  Target  features  were  identified  at  BT3  and  BT4  using  LiDAR,  orthophotography, 
and  hyperspectral  imaging  (HSI)  data  (Sky  Research,  2007b).  No  munitions-related  features 
were  detected  in  the  vicinity  of  the  suspected  75 -mm  air-to-ground  target  in  either  the 
orthophotography  or  the  LiDAR  imagery. 

At  BT3,  the  following  were  detected— an  aiming  circle,  inner  concentric  circles,  a  raised  area 
within  the  aiming  circle  to  the  east,  and  a  fenced  area  extending  outside  the  aiming  circle  to  the 
west  (Ligure  4).  No  crosshair  features  were  mapped,  even  though  some  evidence  for  northern 
and  western  crosshair  arms  is  suggested  in  the  data.  A  possible  ship  target  was  partially  detected 
west  of  the  aiming  circle.  An  area  north  of  the  aiming  circle,  approximately  100  m  x  100  m,  was 
identified  as  being  a  possible  feature  of  interest  related  to  BT3;  this  feature  is  comprised  of  raised 
areas  indicative  of  potential  berm-like  features. 


Figure  4,  Deliueatiou  of  Target  Features  at  BT3  aud  Viciuity, 
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In  the  documented  BT4  area,  an  aiming  circle  1,000  foot  in  diameter  was  mapped;  the  aiming 
circle  was  intersected  by  a  four-armed  crosshair  oriented  NS-EW,  each  arm  extending  from  the 
inner  200-ft  circle  to  about  lOO-ft  past  the  outer  circle.  Four  ship  targets  ranging  in  length  from 
about  160  to  210  m  were  mapped  in  the  vicinity  of  BT4  (Figure  5).  These  features  were 
constructed  by  grading  raised  berms  in  the  shape  of  ship  outlines,  representing  in  approximate 
size  and  shape  a  World  War  Il-era  Japanese  or  German  battleship.  It  is  postulated  that  these 
target  features  at  least  partially  account  for  the  skip  targets  described  in  the  ASR.  These  features 
are  also  best  defined  in  the  analytic  high-pass  FiDAR  image.  One  ship  target  (immediately 
south-southwest  of  the  BT4  circle)  was  observable  in  the  HSI  false-color  minimum  noise 
fraction  (MNF)  and  high-pass  FiDAR  DEM  imagery,  but  not  in  the  orthophotography  or 
hillshade  images  analyzed  in  the  work. 


Figure  5,  Four  Ship  Targets  (Yellow  Outlines)  Were  Detected  in  the  LiDAR  Imagery  in  the 
Vicinity  of  BT4  at  Puehlo  PBR  #2,  (A  number  of  craters  can  be  seen  in  the  imagery 

surrounding  the  ship  targets.) 


4,1,3,2  Borrego 

The  results  of  the  target  feature  analysis  for  the  Borrego  demonstration  included  a  determination 
of  the  exact  size  and  location  of  the  Bomb  Target  64  (BT64)  target  circles,  a  vehicle  target  and 
railroad  strafing  targets.  In  addition,  there  is  evidence  to  support  the  conclusion  that  the  rake 
stations  were  also  used  as  targets;  therefore,  they  are  included  as  targets  that  were  detected  in  the 
data. 

The  target  circles  for  BT64  were  located  in  a  minor  topographic  basin  lying  in  an  upland 
dividing  the  main  branches  of  the  Borrego  Military  Wash  that  bisects  the  study  area  and 
approximately  1,000  ft  to  the  east-southeast  of  the  “target  center”  location  point  provided  in  the 
ASR.  The  BT64  target  feature  consisted  of  three  concentric  circles  approximately  75  ft,  150  ft, 
and  300  ft  in  diameter  (Figure  6). 
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Figure  6,  Extracted  Circular  Aiming  Point  Target  Features  for  BT64  at  Borrego 

Shown  on  Color  Orthophoto, 

These  features  were  best  defined  in  the  orthophotography  from  the  vegetation  pattern  that  has 
emerged  on  the  small  ridges  and  ean  also  be  diseriminated  in  the  LiDAR  analytie  image. 

The  railroad  strafing  target  (Figure  7)  was  located  approximately  2,000  ft  to  the  southeast  of  the 
“mile  long  wooden  track”  location  point  provided  in  the  ASR  in  the  same  topographic  basin 
where  BT64  is  located.  This  feature,  a  raised  berm  less  than  0.5-m  high  and  about  2,500  ft  long 
(about  0.5  mile),  runs  east  to  west  and  was  constructed  by  grading  a  raised  berm  in  the 
configuration  of  a  typical  railroad  bed.  It  was  well  defined  in  the  orthophotography,  LiDAR 
hillshade,  and  analytic  high-pass  LiDAR  images.  Ground  support  field  observations  at  this 
location  included  burned  fragments  of  milled  wood  and  very  high  densities  of  box  nails 
consistent  with  the  ASR  information. 


0  50  100  150  200  Meters 


Figure  7,  Extracted  Railroad  Strafing  Target  at  Borrego  Shown  on  the  Color  Orthophoto, 
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4.1,4  Crater  Detection 


4.1.4.1  Pueblo  PER  #2 

A  total  of  1,103  potential  crater  features  were  mapped  on  the  combined  Phase  I  and  Phase  II 
study  areas;  sixteen  in  the  vicinity  of  BT3  and  more  than  1,000  in  the  vicinity  of  BT4.  No  HE 
cratering  was  evident  in  the  suspected  75 -mm  air-to-ground  target  vicinity. 

4.1.4.2  Borrego 

No  HE  craters  were  detected  in  the  vicinity  of  BT64  or  elsewhere  in  the  demonstration  area.  This 
result  was  not  surprising  since  no  HE  aerial  bombs  had  been  documented  in  use  at  the  site,  and 
the  only  documented  HE  munitions  were  5 -in-high  velocity  air  rockets  (HVAR)  that  normally 
carried  a  relatively  small  (7  pounds  of  trinitrotoluene  [TNT])  HE  charge. 

4.1.5  Range  Infrastructure  Detection 

4.1.5.1  Pueblo  PBR  #2 

Potential  range  infrastructure  features  detected  in  the  EiDAR  and  orthophotography  imagery 
included  transportation  routes,  structures,  and  raised  features. 

4.1. 5.2  Borrego 

Two  rake  stations  and  a  small  depression  were  identified  during  data  analysis.  Positions  for  both 
rake  stations  were  determined  from  the  orthophoto  imagery.  The  western  rake  station  was 
located  approximately  400  m  northeast  of  the  location  provided  in  the  ASR,  and  the  eastern  rake 
station  approximately  330  m  northwest  of  the  ASR  location.  Both  locations  are  situated  on  high 
ground  looking  northwards  down  into  the  area  where  BT64  and  the  railroad  strafing  target  are 
located,  and  each  has  a  clear  view  of  both  target  features.  Transport  features  mapped  include  the 
access  road  through  the  site  that  is  currently  in  active  use  by  park  visitors.  This  road  accesses 
both  the  eastern  rake  station  and  BT64.  A  small  depression  was  also  identified  in  the  data 
analysis. 

4.1.6  Site  Characterization  Results 
4,1,6.1  Pueblo  PBR  #2 

The  results  of  the  EiDAR  and  orthophotography  data  analyses  established  the  precise  location 
and  extent  of  the  two  historically  known  bombing  targets  within  the  study  area  (BT3  and  BT4), 
and  provide  no  evidence  that  a  suspected  75 -mm  target  is  located  within  the  WAA  study  area. 
The  clear  detection  of  the  two  historically  known  bombing  targets,  and  the  conclusive  detection 
of  ship-shaped  skip  bombing  targets  mentioned  but  not  located  in  the  historical  record,  provide 
evidence  that  no  undocumented  similar  targets  exist  within  the  study  area.  While  the  possibility 
for  isolated  munitions  anywhere  in  and  around  the  study  area  cannot  be  eliminated,  the  high- 
certainty  elimination  of  the  majority  of  the  study  area  as  potential  bombing  target  impact  zone 
provides  significant  efficiencies  for  subsequent  survey  activities. 
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4.1. 6,2  Borrego 


The  high  airborne  datasets  confirmed  the  presence  of  several  known  target  features  within  the 
Borrego  Wash  WAA  study  area.  However,  one  target  circle  that  was  identified  from  the  ground 
was  not  detected  in  the  orthophotography  or  LiDAR  datasets  due  to  weathering  conditions  at  the 
site.  No  other  features  indicating  concentrated  munitions  use  were  evident  in  other  portions  of 
the  study  area,  although  it  is  unknown  whether  additional  features  have  been  obscured  by  fluvial 
erosion  processes  or  blowing  sand.  Therefore,  due  to  less  stable  site  conditions  than  at  Pueblo, 
the  reduction  of  munitions  potential  in  non-target  parts  of  the  study  area  is  significantly  less 
certain. 

4.2  PERFORMANCE  CRITERIA 

Table  11  identifies  the  expected  performance  criteria  for  these  evaluations,  complete  with  post¬ 
demonstration  performance  results  (quantitative)  and/or  definitions  and  descriptions 
(qualitative).  Performance  confirmation  methods  included  assessment  of  the  use  of  LiDAR  and 
orthophotography  in  a  high  airborne,  fixed-wing  aircraft;  calculation  of  the  georeference 
accuracies;  and  comparison  of  the  feature  detection  results  to  the  validation  data. 

4.3  DATA  ASSESSMENT 

The  use  of  LiDAR  and  orthophotography  in  a  high  airborne,  fixed-wing  aircraft  is  an  efficient 
means  for  collecting  data  over  large  survey  areas.  The  LiDAR  sensor  and  digital  camera  utilized 
for  this  demonstration  are  easy  to  operate,  although  they  do  require  a  skilled  operator.  Health  and 
safety  requirements  for  the  operation  of  this  technology  are  the  same  as  for  any  flight  operations. 
The  standardized  data  processing  and  analysis  methodologies  are  well  understood  and  easy  to 
use  for  experienced  remote  sensing  analysts. 

Georeference  accuracies  as  specified  in  the  demonstration  design  are  more  than  sufficient  for  the 
accuracy  needed  to  delineate  features  for  further  investigation.  For  PBR  #2,  demonstration  met 
the  expected  georeference  accuracies;  the  BMA  demonstration  did  not.  However,  even  though 
the  georeference  accuracies  were  not  met,  the  ground  survey  team  was  still  able  to  relocate  the 
features  detected  in  the  data  with  sufficient  accuracy  to  verify  their  locations. 

Target  features,  craters,  and  range  infrastructure  features  were  found  to  be  largely  detectable  in 
the  data.  The  validation  data  for  the  PBR  #2  data  confirmed  the  identification  and  location  of 
BT3  and  BT4  and  did  not  find  evidence  of  contamination  in  the  suspected  75-mm  air-to-ground 
target  area,  confirming  the  findings  of  the  LiDAR  and  orthophotography  analysis.  Regarding  the 
detection  of  craters,  the  performance  criteria  originally  developed  for  the  performance 
assessment  were  as  follows:  70%  of  craters  less  than  1  m  in  diameter  were  expected  to  be 
detected  and  90%  of  potential  craters  greater  than  1  m  in  diameter  were  expected  to  be  detected. 
However,  during  the  crater  analysis  conducted  as  part  of  this  project,  the  results  indicated  that 
crater-like  features  smaller  than  1  m  in  diameter  were  not  detectable  in  LiDAR  imagery.  A 
complete  discussion  of  crater  detection  performance  can  be  found  in  the  final  technical 
demonstration  report  (Sky  Research,  2008).  Section  4.5.4  of  the  Pueblo  demonstration  report 
describes  how  the  validation  surveys  determined  the  locations  for  28  ground-observed  circular 
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Table  11,  Performance  Confirmation  Methods  and  Results. 


Type  of 
Performance 
Objective 

Primary  Performance 
Metric 

Expected  Performance 
(Metric) 

Performance 
Confirmation  Method 

Observed  Performance 

Qualitative 

Ease  of  use  and  effieieney 

Effieieney  and  ease  of  use  meets 

General  observations 

Pueblo  PBR  #2:  Pass 

(Primary) 

of  operations 

design  speeifieations 

Borrego:  Pass 

LiDAR: 

Vertieal  aeeuraey  -  15-em 

RMSE; 

Horizontal  aeeuraey  -  40-em 
RMSE 

Pueblo  PER  #2: 
Comparison  of  data  sets 
with  ground  fidueials 

Borrego:  Caleulated  from 
average  offset  for  eight 

Pueblo  PBR  #2: 

Vertieal  aeeuraey: 

Phase  I:  11.1  em 

Phase  II:  3.2  em 

Horizontal  aeeuraey: 

Phase  I:  28.2  em 

Phase  II:  8.5  em 

Quantitative 

(Primary) 

Georeferenee  position 
aeeuraey  for  eaeh  sensor 
system 

eomers  of  two  rake 
stations* 

Borrego: 

Vertieal  aeeuraey:  62  em  (22  em 
relative  to  adjaeent  sample  points); 
estimated  horizontal  aeeuraey  of  89  em 

Orthophotography: 

Horizontal  aeeuraey  -  40-em 
RMSE 

Pueblo  PBR  #2: 
Comparison  of  data  sets 
with  ground  fidueials 

Pueblo  PBR  #2: 

Phase  I:  28.6  em 

Phase  II:  26.8  em 

Borrego:  Caleulated  from 
average  offset  for  eight 
eomers  of  two  rake 
stations* 

Borrego: 

92-em  RMSE 
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Table  11,  Performance  Confirmation  Methods  and  Results  (continued). 


Type  of 
Performance 
Objective 

Primary  Performance 
Metric 

Expected  Performance 
(Metric) 

Performance 
Confirmation  Method 

Observed  Performance 

Quantitative 

(Primary) 

Target  area  deteetion 

>0.90  of  target  areas  having 
topographic  aiming  point 
features 

Comparison  of  ortho  and 
LiDAR  data  analysis  results 
with  ground  validation 
data** 

Pueblo  PER  #2:  BT3  and  BT4 
detected.  No  target  features 
detected  at  the  suspected  75 -mm 
air-to-ground  target  area,  which 
was  confirmed  by  absence  of  target 
evidence  during  validation  surveys. 

Borrego:  0.83 

five  target  features  detected  (target 
circle,  railroad  strafing  target,  two 
rake  stations,  and  small 
depression);  one  target  feature 
(circle)  not  detected 

Quantitative 

(Secondary) 

Crater  detection 

>0.75  (craters  <1  m) 

>0.90  (craters  >1  m) 

Comparison  of  LiDAR  data 
analysis  results  with  ground 
validation  data** 

Pueblo  PER  #2: 

<1  m:  Not  within  detection  limits 
>1  m:  Performance  met 

Borrego:  No  craters  detected 

Range  infrastructure 
detection 

>0.90 

Comparison  of  ortho  and 
LiDAR  data  analysis  results 
with  ground  validation 
data** 

Pueblo  PER  #2:  N/A*** 

Borrego:  Identified  features 
verified  as  range  infrastructure 

*  Performance  confirmation  method  reported  is  typically  comparison  of  datasets  with  ground  fiducials;  however,  for  the  Borrego  demonstration,  another  confirmation  method  (as  described  in  Section 
4. 1.2. 2)  was  used  because  of  restriction  on  emplacements  of  fiducials,  which  also  resulted  in  a  lower  accuracy  than  anticipated. 

**  For  the  Borrego  demonstration,  visual  site  inspection  results  were  utilized  for  validation  purposes. 

***  The  range  infrastructure  visited  during  the  validation  survey  was  not  sufficient  to  calculate  a  meaningful  detection  estimate;  therefore,  no  value  was  reported  for  this  criterion. 
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depressions  identified  as  potential  detonation  eraters,  of  whieh  27  were  deteeted  in  the  analysis. 
Based  on  this  sample,  it  was  estimated  that  the  90%  deteetion  goal  had  been  aehieved.  Last, 
potential  range  infrastrueture  features  were  deteeted  in  the  imagery,  but  it  was  determined  that 
range  infrastrueture  needs  to  be  reviewed  in  the  eontext  of  the  eurrent  land  use. 

The  Borrego  site  presented  a  number  of  interesting  new  challenges  to  WAA  technologies  and 
methods  based  on  a  foundation  of  high  resolution  LiDAR  and  orthophoto  datasets.  First, 
environmental  considerations  required  that  ground-disturbing  fiducials  and  aerial  targets  not  be 
used  led  to  a  reduction  in  geospatial  data  accuracy.  Second,  the  part  of  the  study  area  containing 
the  main  target  features  was  subject  to  water  erosion  and  periodic  flooding  and  wind  erosion  and 
deposition,  with  significant  reduction  in  the  ability  to  observe  features.  As  a  result,  the 
demonstration  did  not  meet  the  performance  objective  of  target  feature  detection  because  one 
feature  out  of  a  total  of  six  possible  features  was  not  detected.  This  feature,  a  second  target  circle, 
was  defined  by  widely  spaced  scattered  rock  piles  aligned  in  a  circular  pattern  that  were  too 
small  to  be  delineated  from  the  underlying  terrain  in  either  the  LiDAR  or  the  orthophoto  datasets. 
The  Borrego  site  was  selected  because  the  erosional  and  depositional  conditions  of  blowing  sand 
and  periodic  flooding  represent  significant  challenges  to  Munitions  Related  Feature  (MRF) 
detection;  it  is  clear  from  the  analysis  that  while  MRFs  could  be  detected  at  Borrego,  false 
negatives  are  possible  in  this  kind  of  environment. 

In  addition  to  natural  disturbance  such  as  the  conditions  encountered  at  Borrego,  the  persistence 
of  munitions-related  features  on  a  site  can  be  impacted  by  human  activities.  At  PBR  #2,  the  lack 
of  crater  features  within  the  BT4  aiming  circle  boundaries  led  to  the  presumption  that  clearance 
and  grading  occurred  sometime  after  HE  bombs  were  dropped.  However,  the  target  area  was  still 
identifiable  due  to  the  extensive  cratering  still  present  surrounding  the  BT4  aiming  circle  and 
adjacent  ship  targets. 

Other  limitations  to  the  use  of  LiDAR  and  orthophotography  technologies  included  delays  due  to 
inclement  weather  during  the  PBR  #2  Phase  I  demonstration. 

4,4  TECHNOLOGY  COMPARISON 

The  WAA  pilot  program  utilized  numerous  sensor  technologies  and  deployment  platforms  in  a 
layered  approach,  from  high  altitude  to  ground-based  systems,  to  explore  how  various 
combinations  of  technologies  might  be  useful  in  efficiently  characterizing  potential  munitions 
contamination  and  supporting  site  decisions  regarding  remediation  activities. 

High-altitude  LiDAR  and  orthophotography  provide  rapid  survey  coverage  over  large  areas  and 
assist  in  the  identification  of  munitions-related  features.  This  rapid,  big-picture  assessment 
provides  important  information  for  defining  the  location  and  extent  of  munitions  use  areas  within 
a  site,  and  provides  support  for  decisions  about  how  to  allocate  more  detailed  and  costly  survey 
efforts  across  the  entire  site. 

The  other  WAA  technologies  employed  in  the  pilot  program,  helicopter-based  magnetometry 
and  ground-based  geophysics,  are  used  to  detect  the  density  and  extent  of  surface  and  subsurface 
ferromagnetic  materials  and  individual  munitions  items,  respectively.  In  comparison  with  high 
airborne  sensors,  these  technologies  provide  more  detail  regarding  the  location  and  abundance  of 
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metallic  munitions  and  artifacts  but  less  information  about  the  patterns  of  vegetation  and  terrain 
disturbance  that  reveal  past  munitions-related  land  uses  and  at  less  efficient  coverage  rates  and 
cost  per  acre.  Therefore,  comparisons  between  the  various  WAA  sensor  technologies  generally 
indicate  complimentary  roles  in  the  overall  WAA  process,  although  some  promising 
technologies  such  as  airborne  imaging  spectrography  and  synthetic  aperture  radar  require  further 
development. 

A  technology  comparison  with  current  practices  that  do  not  incorporate  WAA  remote  sensing 
guidance  in  munitions  site  remediation,  such  as  standard  full-coverage  mag-and-flag  ground 
operations,  highlights  the  value  of  the  WAA  process.  The  orthophotography  and  LiDAR  datasets 
and  analysis  provide  significant  efficiencies  to  subsequent  munitions  recovery  operations  by 
locating  and  characterizing  major  contamination  features  such  as  bombing  targets,  firing  ranges, 
and  other  features.  The  detailed  terrain  and  land  cover  information  generated  also  represents 
important  operational  planning  aids  for  ongoing  recovery  activities. 
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COST  ASSESSMENT 


5.1  COST  REPORTING 

Cost  information  associated  with  the  demonstration  of  all  airborne  teehnology,  as  well  as 
assoeiated  aetivities,  were  traeked  and  doeumented  before,  during,  and  after  the  demonstrations 
to  provide  a  basis  for  determining  the  operational  eosts  assoeiated  with  this  technology.  Tables 
12-14  contain  the  cost  elements  that  were  tracked  and  doeumented  for  this  demonstration.  These 
eosts  inelude  both  operational  and  eapital  eosts  assoeiated  with  the  demonstration  design  and 
planning;  salary  and  travel  eosts  for  support  staff;  equipment  eosts  assoeiated  with  aireraft, 
sensor,  and  eamera;  support  personnel;  and  eosts  assoeiated  with  the  proeessing,  analysis,  and 
interpretation  of  the  results  generated  by  this  demonstration. 

5.2  COST  ANALYSIS 

The  eost  of  an  airborne  survey  depends  on  many  faetors,  ineluding: 

•  Aireraft  costs.  The  rental  rate  for  aircraft  will  be  influeneed  in  part  by  the  type  of 
aireraft  utilized  and  the  eost  of  fuel  (if  ineluded  in  the  eost  of  the  aireraft  time).  In 
addition,  standby  time,  if  needed,  will  inerease  the  survey  eosts. 

•  Length  and  number  of  flight  lines  required  to  survey  the  demonstration  area. 

•  Aeeuraey  requirements,  whieh  influenee  the  speed  and  altitude  of  the  survey 
flights  and  the  amount  of  data  proeessing  required. 

•  Loeation  of  the  site,  whieh  ean  influenee  the  cost  of  mobilization  and  logistics. 

•  Amount  of  analysis  required  to  suffieiently  review  the  data. 

Costs  assoeiated  with  the  pre-survey  ealibration  flights  were  not  ineluded  in  either 
demonstration.  Costs  assoeiated  with  the  ground  validation  surveys  to  eolleet  post-survey  data 
were  not  eonsidered  in  the  eost  analysis,  as  the  validation  was  eonducted  as  part  of  the  WAA 
pilot  program.  Last,  this  demonstration  was  one  of  several  eoncurrent  demonstrations  condueted 
by  Sky  Researeh  for  the  WAA  pilot  program.  Some  of  the  eosts  assoeiated  with  management 
were  shared  among  demonstrations,  lowering  the  costs  incurred  for  this  demonstration. 

Aireraft  eosts  are  a  major  eost  faetor  for  any  airborne  survey.  Signifieant  variables  and  factors 
associated  with  the  mobilization,  data  aequisition,  and  demobilization  eosts  inelude  the  eost  of 
aireraft  time  and  standby  time.  The  eost  of  aircraft  can  vary  depending  on  the  type  of  aireraft  and 
operating  eosts.  Standby  time  can  also  influence  the  eost  of  a  survey  and  is  typieally  assessed  at 
the  cost  of  one  day  of  data  eolleetion,  ineluding  aireraft  eosts,  labor,  and  travel.  For  the  Pueblo 
PBR  #2  demonstration,  two  standby  days  were  ealled  during  the  Phase  I  demonstration  due  to 
weather.  The  standby  time  for  the  aireraft  during  this  demonstration  was  four  hours  of  aireraft 
cost  per  standby  day. 
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Table  12,  Pueblo  PER  #2  Phase  I  Cost  Tracking.^ 


Cost  Category 

Subcategory 

Details 

Costs  ($) 

Start-up  costs 

Pre-Deployment  and  Planning: 
Includes  planning,  contracting, 
pre-survey  site  visit 

Labor:  Contracting 

$2,172 

Labor:  Site  visit 

3,646 

Labor:  Planning 

655 

Site  visit  travel 

1,840 

Materials  and  supplies 

$96 

Subtotal:  predeploymeut 
plauuiug 

$8,409 

Mobilization: 

Personnel  mobilization, 
equipment  mobilization  and 
transportation 

Aircraft  time  (4.5  hours) 

$5,711 

Equipment 

Labor 

1,154 

Travel 

Subtotal:  mobilizatiou 

$7,930 

Total  start-up  costs 

Operatiug  costs 

High  Airborne  Survey:  Data 
acquisition  and  associated 
tasks,  including  aircraft 
operation  time 

Aircraft  time  (5.2  hours) 

Aircraft  standby  time 
(8  hours) 

10,153 

LiDAR  sensor 

1,226 

Digital  camera 

491 

GPS  equipment 

218 

Labor 

2,523 

Travel 

1,251 

Materials  and  supplies 

165 

Total  operating  costs 

$22,627 

Demobilizatiou 

Demobilization: 

Personnel  demobilization, 
equipment  demobilization 
Personnel  demobilization, 
equipment  demobilization  and 
transportation 

Aircraft  time  (4.5  hours) 

$5,711 

Equipment 

0 

Labor 

1,154 

Total  demobilization  costs 

$6,865 

Data  processing 

Data  processing 

Labor 

25,238 

Data  analysis 

Data  analysis 

Labor 

26,045 

Management 

Management  and  reporting 

Project  related  management, 
reporting  and  travel 

30,462 

Subtotal 

$81,745 

Total  Costs 

$127,576 

Acres  Surveyed 

6,700 

Uuit  Cost 

$19.04/acre 

1 


All  costs  reported  for  the  demonstration  include  overhead  and  organization  burden  and  fees. 
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Table  13,  Pueblo  PER  #2  Phase  II  Cost  Tracking,^ 


Cost  Category 

Subcategory 

Details 

Costs  ($) 

Start-up  costs 

Predeployment  and  planning: 
Includes  planning,  contracting, 
pre-survey  site  visit 

Labor:  Contracting 

$300 

Labor:  Site  visit 

5,653 

Labor:  Planning 

2,351 

Site  visit  travel 

3,940 

Materials  and  supplies 

198 

Subtotal:  predeploymeut 
plauuiug 

Mobilization: 

Personnel  mobilization, 
equipment  mobilization  and 
transportation 

Aircraft  time  (4.5  hours) 

$5,711 

Equipment  (4  days  of  GPS 
equipment) 

3,443 

Labor 

7,788 

Travel 

5,946 

Subtotal:  mobilizatiou 

$22,888 

Total  start-up  costs 

$35,330 

Operatiug  costs 

High  airborne  survey:  Data 
acquisition  and  associated 
tasks,  including  aircraft 
operation  time 

Aircraft  time  (4.2  hours) 

$5,341 

LiDAR  sensor 

6,208 

Digital  camera 

3,679 

GPS  equipment 

2,148 

Labor 

2,622 

Travel 

3,239 

Materials  and  supplies 

3,213 

Total  operating  costs 

$26,450 

Demobilizatiou 

Demobilization: 

Personnel  demobilization, 
equipment  demobilization 
Personnel  demobilization, 
equipment  demobilization  and 
bansportation 

Aircraft  time  (4.5  hours) 

$5,711 

Equipment  (3  days  of  GPS 
equipment) 

2,582 

Labor 

2,169 

Travel 

3,291 

Total  demobilization  costs 

Data  processing 

Data  processing 

Labor 

Data  analysis 

Data  analysis 

Labor 

8,407 

Management 

Management  and  reporting 

Project  related  management, 
reporting  and  travel 

29,137 

Total  costs 

$134,494 

Acres  surveyed 

6,600 

Uuit  cost 

$20.38/acre 

2 


All  costs  reported  for  the  demonstration  include  overhead  and  organization  burden  and  fees. 
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Table  14,  Borrego  Military  Wash  Cost  Tracking.^ 


Cost  Category 

Subcategory 

Details 

Costs  ($) 

Start-up  costs 

Predeployment  and  planning: 
Includes  planning, 
contracting,  pre-survey  site 
visit 

Labor 

$3,413 

Travel 

$5,546 

Mobilization: 

Personnel  mobilization, 
equipment  mobilization  and 
transportation 

Aircraft  time  (4.2  hours) 

$5,282 

Equipment 

Labor 

Travel 

2,226 

Subtotal:  moblllzatlou 

Total  start-up  costs 

Operatiug  costs 

High  airborne  survey:  Data 
acquisition  and  associated 
tasks,  including  aircraft 
operation  time 

Aircraft  time  (3.6  hours) 

LiDAR  sensor 

6,208 

Digital  camera 

1,738 

GPS  equipment 

384 

Labor 

1,783 

Travel 

1,867 

Materials/suppbes/shipping 

1,998 

Total  operatiug  costs 

Demobilizatiou 

Demobilization: 

Personnel  demobilization, 
equipment  demobilization 
and  transportation 

Aircraft  time  (3.4  hours) 

Equipment 

Labor 

Travel 

Total  demobilizatiou 

Data  processiug  aud 
aualysis 

Data  processing  and  analysis 

Labor 

Total  data  processiug  aud  aualysis 

$11,400 

Mauagemeut 

Project  related  management, 
reporting  and  conbacting 

Labor 

$14,834 

Total  mauagemeut 

$14,834 

Total  costs 

$66,618 

Acres  surveyed 

7,940 

Uuit  cost 

$8.39/acre 

3 


All  costs  reported  for  the  demonstration  include  overhead  and  organization  burden  and  fees. 
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Mobilization  and  demobilization  costs  are  most  significantly  a  function  of  the  distance  from  the 
home  base  for  the  aircraft.  In  addition  to  the  cost  of  mobilizing  and  demobilizing  the  aircraft,  the 
cost  of  mobilizing  equipment  (sensors  and  GPS  equipment)  can  add  significantly  to  costs.  For 
these  demonstrations,  the  daily  rates  for  the  LiDAR  sensor  and  digital  camera  were  not  charged 
for  mobilization  and  demobilization  because  each  required  less  than  one  day.  For  a  mobilization 
taking  a  full  day  or  longer,  the  daily  rate  for  the  LiDAR  sensor  and  digital  camera  would  have 
been  assessed,  increasing  the  mobilization  and  demobilization  costs.  GPS  equipment  was 
mobilized  from  Denver,  and  the  preparation  and  shipping  time  was  charged  for  mobilization  and 
demobilization.  Therefore,  for  a  site  requiring  a  longer  mobilization  distance,  the  mobilization 
and  demobilization  can  take  up  a  correspondingly  larger  amount  of  the  budget,  especially  when 
considered  as  a  percentage  of  the  cost  for  surveys  of  relatively  small  areas. 

For  the  costs  associated  with  the  data  acquisition  flights,  the  predominant  costs  are  the  cost  of  the 
aircraft  and  equipment.  The  major  driver  for  these  costs  is  the  survey  size;  secondarily,  the  shape 
of  the  survey  areas  can  influence  the  costs.  Fixed-wing  aircraft  can  cover  up  to  approximately 
15,000  acres  per  day  on  average.  Sites  less  than  this  in  size  will  therefore  be  more  expensive  on  a 
per  acre  basis.  With  respect  to  the  shape  of  the  survey  boundaries,  the  demonstrations  at  Pueblo 
PBR  #2  and  Borrego  were  completed  efficiently  in  part  because  the  survey  areas  were  regularly 
shaped,  allowing  efficient  coverage  with  a  minimum  number  of  flight  lines.  Irregularly  shaped 
survey  areas  or  surveys  with  a  number  of  noncontiguous  parcels  will  increase  the  number  of 
flight  lines  required  for  complete  coverage,  thereby  increasing  flight  time  and  survey  costs.  The 
remainder  of  costs  associated  with  data  acquisition  include  labor,  travel,  and  materials/supplies. 

Data  processing  and  analysis  costs  are  generally  linear  with  project  size.  For  these 
demonstrations,  data  processing  and  analysis  made  up  approximately  31%  of  the  costs  associated 
with  the  Pueblo  PBR  #2  demonstrations.  The  data  processing  and  analysis  functions  for  the 
Borrego  demonstration  made  up  approximately  17%  of  the  demonstration  cost;  this  is  due  in 
large  part  because  no  additional  WAA  data  was  collected  for  this  site,  which  decreased  analysis 
time.  Last,  the  site  conditions  encountered  at  Pueblo  PBR  #2  and  Borrego  were  benign  for 
analyzing  the  data.  Environmental  conditions  such  as  steep  topography  or  vegetation  can 
increase  the  amount  of  time  needed  for  analysis  in  more  challenging  settings.  An  additional  cost 
factor,  processing  and  analysis,  has  been  decreasing  with  experience  at  multiple  sites,  automation 
of  processing  and  analysis  routines,  and  increased  computing  power  resulting  in  faster 
processing. 

Project  management  and  reporting  were  a  somewhat  significant  cost  for  these  demonstrations,  as 
the  projects  were  conducted  under  the  WAA  pilot  program  and  required  more  meetings  and 
reporting  than  would  generally  be  expected  for  a  production  level  survey. 

5.3  TYPICAL  AIRBORNE  SURVEY  COSTS 

Mobilization  distance,  site  size  and  shape,  site  conditions,  and  project  objectives  can  influence 
the  costs  of  data  collection  and  analysis.  In  addition,  this  demonstration  utilized  a  fixed-wing 
aircraft.  Different  kinds  of  fixed-wing  aircraft  will  have  somewhat  different  hourly  rates  than  the 
rates  used  in  this  demonstration. 
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To  generalize  typieal  airborne  survey  eosts,  various  seenarios  are  presented  in  Table  15  for 
several  sizes  of  survey  sites.  For  these  seenarios,  the  following  assumptions  have  been  made;  the 
site  is  generally  amenable  for  the  deteetion  of  surfaee  features  in  remote  sensing  data  and 
therefore  will  require  an  average  amount  of  proeessing  and  analysis;  the  projeet  objeetives 
require  aeeuraeies  similar  to  those  used  for  these  demonstrations;  the  mobilization  distanee  is  a 
4-hour  or  less  flight  distanee;  and  for  a  very  small  site  (~1,000  aeres),  the  ground  support  team 
will  be  one  person  traveling  with  the  survey  team. 

Eaeh  seenario  ineludes  several  eategories  of  eosts  and  tasks,  ineluding  the  following: 

•  Planning,  Preparation  and  Management 

Historieal  information  review 

Work  plan  development 

Flight  planning 

Logisties  planning 

Ground  eontrol  planning 

Staff  preparation  and  equipment  review 

Management  ineluding  eontraeting,  elient  interaetion,  ete. 

•  Mobilization/Demobilization 

4  hours  of  fixed-wing  flight  time  eaeh  direetion 

Labor  for  pilot,  eo-pilot,  sensor  operator,  ground  survey  team 

1  day  of  equipment  eosts  (LiDAR  sensor,  eamera,  GPS  equipment) 

Travel  eosts 

•  Data  Aequisition 

Fixed-wing  aireraft  (minimum  of  4  hours  of  flight  time  per  day) 

Materials  and  supplies  for  ground  fidueials 

Labor  for  pilot,  eo-pilot,  sensor  operator,  ground  survey  team 

Travel  eosts 

•  Data  Proeessing,  Analysis,  and  GIS  Produets 

Data  proeessing 
Analysis 
Quality  eontrol 

GIS  produets,  ineluding  maps  (ineluding  online  mapping  for  larger  sites) 

•  Reporting/Doeumentation 

Pinal  report 
Metadata  development 
Data  eompilation  and  delivery 
Data  arehiving 
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Table  15,  Estimated  Costs  Scenarios  for  LiDAR  and  Orthophotography. 


Cost  Category 

1,000- 
Acre  Site 

10,000- 
Acre  Site 

50,000- 
Acre  Site 

250,000- 
Acre  Site 

Planning,  Preparation,  and  Management 

$10,000 

$16,000 

$24,000 

$31,000 

Mobilization/Demobilization 

15,000 

31,000 

32,000 

39,000 

Data  Aequisition 

30,000 

60,000 

89,000 

251,000 

Data  Processing,  Analysis,  and  GIS  Products 

12,000 

22,000 

49,000 

178,000 

Reporting  and  Documentation 

2,000 

5,000 

10,000 

20,000 

Total  costs 

$69,000 

$134,000 

$204,000 

$519,000 

Costs  per  acre 

$69.00 

$13.40 

$4.08 

$2.08 

As  discussed,  additional  costs  would  be  assumed  for  a  greater  mobilization  distance;  each 
additional  hour  of  flight  time  would  increase  the  mobilization/demobilization  costs  by 
approximately  10%.  Data  acquisition  costs  could  decrease  if  the  required  accuracies  are  less 
than  those  used  for  these  demonstrations,  as  the  flight  survey  speeds  could  increase.  Data 
processing  and  analysis  costs  could  also  decrease  from  the  estimates  provided  if  less  extensive 
data  analysis  and  feature  selection  is  required.  The  deliverable  products  include  reports,  maps, 
GIS  data  layers,  and  in  some  cases,  an  online,  interactive  mapping  site  that  allows  stakeholders 
access  to  spatial  data  for  viewing,  exploring,  analysis,  and  display.  In  terms  of  basic  data 
products,  these  cost  estimates  include  full  LiDAR  point  clouds  in  Log  ASCII  Standard  (LAS)  or 
ASCII  format,  surface  and  bare  earth  DEMs,  high-resolution  orthophotography  (both  image  tiles 
and  geodatabase  mosaics),  and  all  feature  extractions  in  vector  feature  classes  in  a  GIS 
geodatabase.  All  data  collection,  processing,  calibration,  and  accuracy  reporting  is  also  included. 
The  estimates  above  assume  that  for  the  10,000+  acre  sites,  an  online  mapping  site  will  be 
created  to  support  client  interaction.  The  costs  for  the  GIS  products  may  be  somewhat  lower 
without  online  mapping.  Last,  the  reporting  for  LiDAR  and  orthophotography  projects  may  vary 
according  to  the  requirements  of  the  client. 

5.4  COST  CONCLUSIONS 

A  number  of  factors  should  be  considered  for  DoD-wide  application  of  WAA,  including  the 
acquisition  of  LiDAR  and  orthophotography  data,  when  evaluating  the  appropriateness  of  the 
airborne  technologies  and  potential  for  cost  savings.  Sites  must  be  large  enough  to  justify  the 
deployment  of  aircraft  and  equipment  to  conduct  a  survey.  Climatic  conditions  and  terrain  can 
limit  the  results  of  surveys.  Time  of  year  should  also  be  taken  into  account  with  respect  to  sun 
angles,  leaf-off,  and  likelihood  of  snow  (obscuring  the  earth’s  surface)  or  other  weather 
impediments.  However,  in  amenable  sites,  the  use  of  LiDAR  and  orthophotography  to  focus 
subsequent  helicopter  magnetometry  and  ground  surveys  can  provide  substantial  cost  savings. 
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6.0  IMPLEMENTATION  ISSUES 


6.1  COST  OBSERVATIONS 

The  costs  as  reported  and  analyzed  are  for  the  concurrent  collection  of  orthophotography  and 
LiDAR  data.  Remote  sensing  analysts  utilized  the  LiDAR  data  more  extensively  for  detection  of 
munitions-related  features;  orthophotography  was  used  to  cross-reference  and  verify  some  of  the 
features  detected.  However,  a  few  features  were  better  defined  in  the  orthophotography.  Because 
the  two  data  sets  can  be  collected  concurrently,  the  cost  to  acquire  orthophotography  with 
LiDAR  data  does  not  significantly  add  to  the  overall  cost.  The  costs  associated  with  processing 
orthophotography  data  are  on  the  order  of  25%  of  the  total  data  processing  budget.  The 
usefulness  of  orthophotography  in  corroborating  visual  analyses  of  LiDAR  data  supports  the 
recommendation  that  these  data  sets  together  provide  the  greatest  benefit  for  WAA. 

Overall,  these  sites  were  amenable  to  cost-effective  collection  of  high-altitude  data:  they  had 
relatively  short  mobilization  distances;  they  were  open  areas;  and  the  demonstration  boundaries 
provided  an  aerial  footprint  for  the  efficient  collection  of  data.  For  these  demonstrations,  the  sites 
were  relatively  small  for  the  use  of  these  technologies  (i.e.,  under  10,000  acres,  requiring  less 
than  a  full  day  for  data  collection).  The  costs  associated  with  larger  surveys  will  be  less,  although 
mobilization  costs  may  be  greater  if  the  mobilization  distance  is  greater. 

Sky  Research  has  been  able  to  reduce  some  of  the  costs  associated  with  collecting,  processing, 
and  analyzing  LiDAR  and  orthophotography  data  by  incorporating  recent  lessons  learned  from 
subsequent  munitions  response-related  projects.  For  example,  as  project  organization  activities 
have  become  more  standardized  and  efficient  there  has  been  a  reduction  in  the  amount  of  time 
required  in  planning  large  scale  surveys.  The  addition  of  supplementary  computing  resources 
(both  hardware  and  software)  in  2006  and  the  development  of  standard  processing  models  and 
algorithms  have  led  to  an  increase  in  automation  and  a  decrease  in  the  amount  of  time  and  labor 
required  to  process  large  data  sets.  Lastly,  since  the  Phase  I  data  collection  at  Pueblo  in  2004, 
there  have  been  improvements  in  sensor  technologies  yielding  increased  efficiency;  advances  in 
these  technologies  will  likely  continue  to  lower  the  costs  associated  with  their  use. 

6.2  PERFORMANCE  OBSERVATIONS 

The  performance  objectives  were  largely  met  by  this  demonstration.  The  georeference  accuracies 
for  both  data  sets  were  higher  than  expected  for  the  Pueblo  demonstration.  Feature  detection  for 
the  Pueblo  demonstration  met  the  expected  performance.  Target  circles,  ship  targets,  craters,  and 
range  features  were  all  detectable  in  the  imagery.  The  validation  data  confirmed  the 
identification  and  location  of  BT3  and  BT4  and  did  not  find  evidence  of  a  target  area  in  the 
suspected  75 -mm  air-to-ground  target  area,  confirming  the  findings  of  the  LiDAR  and 
orthophotography  analysis. 

Borrego  presented  a  more  challenging  site  because  site  restrictions  and  climatic  conditions 
obscured  some  features  and  one  of  the  two  target  areas  was  not  detected.  Therefore,  it  is 
important  to  review  and  assess  the  site  characteristics  and  the  potential  impact  these  conditions 
might  have  on  the  results  before  undertaking  a  WAA. 
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6.3  SCALE-UP 


There  are  no  scale-up  issues  with  this  technology;  LiDAR  and  orthophotography  can  be  utilized 
as  demonstrated  to  characterize  a  large  number  of  sites.  However,  their  use  on  very  small  sites 
may  not  be  cost  effective;  the  larger  the  site,  the  more  cost  effective  their  use  on  a  per  acre  basis. 

6.4  OTHER  SIGNIFICANT  OBSERVATIONS 

The  two  sites  illustrate  the  difference  site  conditions  can  have  on  results.  The  Pueblo  site  was  a 
good  candidate  for  the  use  of  high-altitude  remote  sensing  technologies  for  site  characterization 
as  surface  features  persisted  and  were  detectable  in  both  the  LiDAR  and  orthophotography 
imagery.  The  Borrego  site  qualifies  the  generalization  that  arid  desert  and  high  plains  munitions 
sites  are  uniformly  well  preserved  from  natural  disturbance  and  amenable  to  remote  sensing 
feature  detection  methods.  Soils,  hydrology,  climate,  and  human  disturbance  factors  need  to  be 
carefully  considered  for  each  prospective  WAA  site  in  order  to  understand  the  probable 
persistence  of  features  over  time  and  the  likelihood  of  detection  for  various  types  of  munitions- 
related  features. 

6.5  LESSONS  LEARNED 

The  primary  benefit  of  these  high-altitude  technologies  are  the  rapid  characterization  of  large 
open  areas.  Cost  analysis  shows  that,  in  general,  costs  per  acre  decrease  with  the  increase  in  size 
of  the  project  area.  However,  complex  sites  (e.g.,  densely  vegetated  or  topographically  steep  and 
variable  landscapes)  requiring  extended  labor  in  data  processing  and  analyses,  or  complicated 
data  collection  surveys  (due  to  inclement  weather,  climatic  conditions,  logistical  difficulties,  etc.) 
can  increase  the  per  unit  costs. 

6.6  END  USER  ISSUES 

Implementing  WAA  for  production-level  surveys  should  include  end  users  in  the  project.  For 
this  project,  ESTCP  utilized  the  WAA  Advisory  Group  to  understand  and  evaluate  potential  end 
user  issues  and  concerns  that  can  impact  the  widespread  implementation  of  WAA  technologies. 
End  users  should  also  be  provided  online  access  to  WAA  data  and  analytical  tools  through  the 
use  of  GIS,  as  demonstrated  in  the  WAA  GIS  demonstration  (MM-0537).  East,  the  DEM  and 
orthophotography  datasets  can  be  utilized  for  site  planning  across  the  entire  site  for  a  variety  of 
purposes  and  should  be  provided  to  end  users. 

6.7  APPROACH  TO  REGULATORY  COMPLIANCE  AND  ACCEPTANCE 

The  ESTCP  Program  Office  established  an  Advisory  Group  to  facilitate  interactions  with  the 
regulatory  community  and  potential  end  users  of  this  technology.  Members  of  the  Advisory 
Group  included  representatives  of  the  USEPA,  State  regulators,  USAGE  officials,  and 
representatives  from  the  services.  ESTCP  staff  worked  with  the  Advisory  Group  to  define  goals 
for  the  WAA  pilot  program  and  develop  Project  Quality  Objectives. 

A  number  of  issues  will  need  to  be  overcome  to  allow  widespread  implementation  of  WAA 
beyond  the  pilot  program.  Most  central  is  the  change  in  mindset  that  will  be  required  if  the  goals 
of  WAA  extend  from  delineating  target  areas  to  collecting  data  that  are  useful  in  making 
decisions  about  areas  where  there  is  not  indication  of  munitions  use.  Therefore,  the  challenge  for 
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adoption  of  a  WAA  approach  with  respect  to  regulatory  aeeeptance  may  be  eollecting  sufficient 
data,  evaluating  the  applieability  of  these  teehnologies  to  uneontaminated  land,  and 
understanding  the  results.  Similarly,  demonstrating  that  WAA  data  ean  be  used  to  provide 
information  on  target  areas  regarding  boundaries,  density  and  types  of  munitions  to  be  used  for 
prioritization,  eost  estimation,  and  planning  will  require  that  the  error  and  uncertainties  in  these 
parameters  are  well  understood. 
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